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ABSTRACT 


Face (expression) recognition (FR) is a very exciting problem in the area of image processing and computer vision. Any 
FR system should be savvy to unequivocally detect facial images. Several kinds of features have been considered for face 
(expression) recognition in past years. It is noticed that some of the simple aggregate statistical features have not attracted 
the researchers for face (expression) recognition problems. In this paper, our contribution is about feature ordering and 
discussing the ability of distinguishing and non-distinguishing images. This article explores the ability of representing and 
classifying facial images through some aggregate statistical features such as mean, standard deviation (Std), Coefficient of 


Variation (Cv), and 7 invariant (spatial) moments. 
KEYWORDS: ANOVA, Mean, STD, CV, IM, Post-Hoc Analysis, Invariant (Spatial) Moments 
INTRODUCTION 


The interest in the face (expression) recognition has been growing greatly during the past two decades due to the need for 
more secure ways of protecting information for both corporate and federal interests, and many more. With the invention of 
Digital Camera Technology and Internet Photo Sharing sites, the biometric facial recognition technology is now utilized in 
everything right from surveillance to targeted marketing. Today, many industries are getting assistance with facial 
recognition system. Like in taking logs, the official record of events, computer entertainment, virtual reality, multimedia, 
database retrieval, information security for example. operating system, medical records, online banking., automated border 
controls, personal security driver monitoring system, Forensic applications, passport, driver’s license, the desire to 


development of human-computer interface (expression)s, interactive movies and games, home video surveillance system. 


Computer and telephone companies are providing more layers of biometric security to customers, Law 
enforcement agencies using face (expression) recognition system to keep the public safer, in the investigation, in identity 
verification for example, in the year 2011, using facial recognition system confirmed the identity of Osama Bin Laden after 
he was killed in the USA. theraid, Airports, and metro station authorities improving travelers’ security and convenience, 
big commercial companies have used facial recognition technology to draw attention and promote their sales. For example, 
in the year 2009, Coca-Cola Zero launched a Facial Profiler App on Face (expression)book that scanned photos for people 


who looked like you. In the year 2012 Suhas et al (2012) developed a face (expression) recognition system using Principal 


Component Analysis and Linear Discriminant Analysis. 


Automatic facial detection and recognition system is an active research area bridging other disciplines like 
machine learning, image processing, pattern recognition, artificial intelligence, biometrics and computer vision. This is 
represented in the Figure 1. Face (expression) recognition system is a necessary first step in many applications such as 


human computer interface (expression), facial expression recognition, and gender recognition [6]. 
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Figure 1: Interconnection of Face (Expression) Recognition and Other Fields of Study. 
LITERATURE REVIEW 


In the late 19th century, Alphonse Bertillon, a police official in Paris, developed a manual database by storing photos of the 
suspect’s full face (expression), profile with name, bodily measurements, and other information. He tried to identify 
criminals with accurate measurements. He also published guidelines to measure body parts and classified information. This 
was soon adopted by police worldwide. Later, the law enforcement started using photographs of wanted conspirators onto 
posters. In cases, where no photo of the suspect was available, the police were dependent on hand-drawing of a suspect’s 


face (expression) Leone (2021). 


In 1960, Woodrow Wilson Bledsoe, a pioneer of artificial intelligence introduced semi-automated computer-based 
facial recognition. He devised a system for manually noting key facial benchmarks on each image like the width of the 
mouth, the distance between eyes, nose etc. These metrics were inserted into a database. Then, when a new face 
(expression) photograph of an individual was given, the Bledsoe’s system was able to retrieve the face (expression) from 


the database that most closely matched. 


In the 1970s, the researchers Goldstein, Harmon, and Lesk were able to add increased accuracy of manual facial 
recognition system by including 21 specific key facial landmarks like lip thickness and hair color in order to identify face 
(expression)s automatically. The Kanade feature-based recognition system developed in1973 is one of the first automated 
face (expression) recognition systems. A work by Kelly (1970) on visual identification of people by computer related to 


automatic face (expression) recognition was also carried out at Stanford University. 


In 1980s and 1990s researchers continued working in the field of Biometrics. In 1988, the research by 
mathematicians Kirby and Sirovich (1990) at Brown University applied linear algebra to facial recognition called Eigen 
face (expression)s. They were able to show that feature analysis on a collection of facial images could form a set of basic 
features. They were able to prove that less than one hundred values were required in order to accurately code a normalized 
face (expression) image. In 1991, computer scientists Matthew Turk and Pentland (1991) at MIT expanded upon the Eigen 
face (expression)s approach by discovering how to detect face (expression)s within images using technology and 


environmental factors leading to the automatic facial recognition system. 


From 1993 to 2000 the Defense Advanced Research Projects Agency (DARPA) and the National Institute of 
Standards and Technology (NIST) created a database of facial images. In 2003 the database was updated to include high- 
resolution images. In 2014 the FBI under state-of-the-art facial recognition technology replaced its old fingerprinting 
system with the world’s biggest biometric database including voice features, palm prints, and even DNA profiles by 
combing civil and criminal information within one master database. It allowed them to compare the suspect information 
with a large database of facial images collected from digital camera devices seized under a search warrant, employment 
background checks, surveillance cameras, mug shots, etc. to find a correct match. The facial recognition system setup 
consists of advanced cameras that capture photos of people who pose or simply walk by, and sophisticated 
software working on those pictures will attempt to find the right match from this gigantic database to identify the person(s) 
in the image. The interest in the face (expression) recognition has been growing greatly during the past two decades due to 


the need for more secure ways of protecting information for both corporate and federal interests and many more. 
PROPOSED MODEL 


A randomly selected sample of 10 people images from BioIDFace (expression) Database with labels as given in Figure 2. 
The following section 1 gives the descriptive statistics of features and identifying the significant features using ANOVA. 
The number of treatments can be as many as distinct persons in the image pool. The extracted feature data can be viewed 


as multivariate data with several treatments. 


In section 2 the conditional distribution of features/ ordering statistically significant features has been computed 
and hence an attempt form rule for distinguishing classes. A hierarchical clustering analysis has been carried out in section 


3with Post-hoc analysis on each of the features and conclusions discussed. 





Figure 2: Sample Face (Expression)s of Selected 10 Individuals. 


Descriptive Statistics and ANOVA 


Ten features such as mean, standard deviation, coefficient of variation (Cv) and seven invariant moments IM1, IM2, IM3, 
IM4, IM5, IM6, IM7 descriptive statistics were extracted from test sample of 1Oimages. An analysis of variance was carried 
out on all extracted features by treating image labels as treatments and their feature values as responses to see which of 


these features is significant. 


A detailed descriptive statistical analysis was carried out on this multivariate sample. A sample of descriptive 


statistics on 4 features is shown in Table 1. 


Table 1: Descriptive Statistics Sample of Mean, Std, CV, and IM1 
Descriptive Statistics 


po Mer Std | IM1 


PN Valid 445 
N = 
Missing | OD TCC eC 
00136060037 
000007739106 
00132788600 
001135418" 
000163256624 
1.362 
116 
1.671 
231 
10 00118799580 
20 00123061540 
25 00123663550 
30 00126492640 
40 00130659320 
Percentiles: 50 00132788600 
00135116780 
70 00138998880 
75 00141607000 
00143902980 
00159126880 


An analysis of variance is performed on feature by feature and the results are provided in Table 2 and then 





followed by their feature distribution Histograms. 


Table 2: Analysis of Variance of Each of the Feature 
ANOVA 


Sum of Squares eens Mean Square F: Significance: 
freedom 


| between groups> | 46366.135, | 9 | 5 151.793 | 46.355 
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Table 2: Contd., 


| between groups> [| 000 | S| S000 Ss] 48.9906 =| 00S 
| within groups> [| 000,“ | 485 | | 
| total> | O— “=e; 
| between groups> [| 000 | |S 00S] 120491 | 00S 
| within groups> [| 000,—=$s | 485 | | 
| total> | 00 

| between groups> [| 000 

| within groups> | 000 

total> 

| between groups> [| 000 | | S000 —Ss| 20.291 | 00S 
| within groups> [| -000,—($s | 485 | | 
| total | O—“=e]| 
| between groups> [| 000 | |S 00-S—s| 8060 S| 
| within groups> [| 000,—“$s | 485 | | 
| total | | 
| between groups> | 000 | |S 00S] 2143 | 00S 
| within groups> [| 000,—#s | 485 | | 
| total> | O— “=e 
| between groups> [| 000 | |S 0—s| 4S | 00S 
| within groups> [| 000, = | 485 | | 
| total | O— $e | 


A univariate one-way analysis of variance on the sample image classes for each of the feature in Table 2 shows 





that the F-statistic is higher and their p value smaller than 0.05 or 0.01. We conclude that the considered features mean, 
Std, Cv, and the seven Invariant Moments (IM1, IM2....1M7) are significant and they contribute or consist information 


about the distinguishing facial image classes. 


The histograms of features are displayed in the following Figure: 3.We see no outliers. All features are roughly 


normally distributed. The distribution of IM3 and IM4 data is nothing like a normal distribution. 
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Figure 3: Histograms of Features. 


The significant features recognised from ANOVA table are arranged in the decreasing value of F statistic. This is the 
feature ordering. The features are now ranked as given in table 3. Hence, we assume that these features are potential 


enough to distinguishing and non-distinguishing image classes. 


Table 3: Ranked Features 


F-statistic 
from 
ANOVA 


pum | oan | 2 


Feature 
name 


| mi | 48.996 | 3 

| mean | 46.355 | 4 

| cy | 367 | 

pia | 7276 | 8 

po | 47s | 9 
IMS 


3.06 10 








The feature with highest F value will be routing into the Image classes distinguishing model first, then the second 


feature with the next highest F value, and so on. 
POST-HOC ANALYSIS 


Post hoc tests are an integral part of Analysis of Variance (ANOVA) and useful to analyze the results of the experimental 
data. Since all the selected features are significant among 10 image classes, a post-hoc analysis is carried on eachfeature 
data to see which pair of image classes differ significantly under each feature. A post-hoc analysis is carried out on each 
feature. A sample of post-hoc pair wise statistics analysis for the top ranked significant feature variable “std” on the image 


class 180 with each of other classes 1s given in table 4 below. 


Table 4: Post-HOC Test Results on Feature’ Std’’ 


evendent (D (J) Std. 95% Confidence Interval: 
‘ Mean Difference (I-J): Lower Upper 
Variable: Class: Class: Error: 
Bound: Bound: 


5.60412497738* 0.684562 | 0 | 4.258665 6.949585 
-3.33141414987* 0.603352 | 0 |  -4.51726 -2.14557 
-3.11159204208* 0.767172 | 0 |  -4.61942 -1.60377 


| 449 | -3.29872666656* | 0.753678 | 0 | -4.78003_ | -1.81742_| 
180 | 483 | -10.24388442845* | 0.735882 | _ 0 | -11.6902__|_-8.79756_| 
| 519 | _3.64732255810* | 0.730507 | 0 | 2.21156 | 5.083086 _| 
| 660 | —_-8.00676296579* | 0.691472 | 0 | 6.647722 | 9.365804 _| 
| 741 | (10.05779259221* | 0.725379 | 0 | 8.632109 | 11.48348 | 


We noticed that the “std” has successfully distinguished the class 180 from all other classes. The labelled class 
180 is significantly differing with 260,322,418, 449, 483, 519, 660, 741 and 973classes. Thus the feature “std” will 





contribute information about the image distinguishing and can be the part of the model of face (expression) recognition. 


Another sample of post-hoc test on the significant dependent variable “mean” discussed below. The class of 180 


with all other classes is given in table 5 below. 


Table 5: Post-HOC Test Results on Feature’ Mean’ 


pebeucent oO (J) Mean Difference (I-J): a ig: pon a 
Variable: | Class: | Class: ; Error : Upper Bound: 
| 260 | -18.15018680446 | 2.227444 | 0 | 22.5281 | -13.7722958 
| 322 | -19.52397918021 | 1.963201 | 0 | -23.3825_ | -15.6654411 
| 418 | -30,22499182496 | 2.496242 | 0 | -35.1312_ | -25.3187959 
| 449 | -32.01614348723 | 2.452335 | 0 | 36.836 | -27.1962442 
Mean: 180 | 483 | -34.65693781071___|_2.394431 | 0 | -39.363_ | -29.9508464 


-25.056348 18346 2.376944 | O | -29.7281 -20.3846262 
| 660 -4.1201 2.249928 | 0.068 -8.54218 0.301983838 


-24.54972619712° 2.360257 | O | -29.1887 -19.9107996 
-16.78028503095° 2.300494 | O | -21.3018 -12.2588187 





As we noticed the individual labeled 180, is significantly differing with all other labeled classes except 660. In 
other words, the feature “mean” is able to distinguish the class 180 from 260, 322, 418, 449, 483, 519, 741, and 973. The 


images 180 and 660 have the same mean. Therefore, it means there will be some classes with equal mean among the image 





classes. Thus this feature may not contribute clear information in distinguishing image classes even though is proved to be 


a significant feature in ANOVA. This feature may be considered in building the model. There is a need to probe further in 


this direction. 


MEAN PLOTTING 


A visual display of the feature means plots help us to visualize our results and therefore included them in our write up. We 


can see increases and decreases in mean values of each feature. For example, in first mean plot of data it shows that label 


519 and 741 are not clearly distinguishable with respect to mean of means. 
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Figure 4: Mean Plots of Features. 


We demonstrated the possibility of ordering the features using F-statistics in this article. In our example, the 
feature “mean” is not able to distinguish two image classes in post-hoc analysis because they have the same mean. Thus 
there is a need to elicit the knowledge to distinguish image classes that do not differ significantly. Thus there is a need for 
probing additional knowledge to distinguish the image classes. Simply confining to this procedure of extracting significant 


features for distinguishing the image classes is insufficient. 
CONCLUSIONS 


We prove our statistical features significance, not by using axiomatic approach, not by an experimental approach, not by 
using modeling, or not by simulation approach. We are analyzing and showing using data itself. This is a data science 


approach. Thus our work in this article starts with the exploratory data analysis on the benchmark BioIDface (expression) 


data set. 


An illustration of the extraction of features, their descriptive statistics on a test sample of 10 images are given. An 
analysis of variance was carried out on all extracted features to know which of these features are significant. The 
significant features are then ranked. A post-hoc analysis is carried on each feature to see which pair of image classes differ 
significantly under each feature. Simply confining to this procedure of extracting significant features for distinguishing the 


image classes which is insufficient is established in this article. 
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